Node.JS

Node.JS, NodeJS or just Node, can be used for more then just web development, you can for example use nw.js (nodeJS + Webkit) to make cross-platform desktop applications, or make an Internet daemon. Remember, Internet is more then just port 80!
Most people use NodeJS for web development though, and I can't blame them for doing so. But NodeJS and JavaScript doesn't work well for rendering HTML code. Actually intertwining HTML with your server logic is considered bad practice, even though you can get away with it using frameworks like PHP or classic ASP.

How to build web applications using Node.JS

If you have some experience in web development you should be familiar with this sort of software stack:
Apache -> PHP -> Database

But in Node.JS it's usually just:
Node.JS -> Database

And that's where Node.JS becomes a bit too complicated. Sure it has a fast, built-in and lightweight http-server module. But because it's so lightweight you are either required to write a lot of boilerplate code for serving the requests or go with a popular framework like express.
That's not the optimal way of doing it though. Instead, you should try to separate concerns. Let something like nginx handle the requests and proxy to your Node.JS application.

Nginx

Make all browser client files static and let nginx serve them.
Then have the client open up a websocket that nginx proxies to the Node.JS script.
This can be achieved with the following nginx configuration:

server {
  listen 80;
  #listen [::]:80 ipv6only=on;
  #listen 443 ssl;

  server_name joha.nz;
  server_name www.joha.nz;

  server_tokens off;

  root /tank/www/joha.nz/;
  index index.html index.htm;

  location / {
    charset  utf-8;
    try_files $uri $uri/ =404;
  }
  
  location /ws {
    # Upgrade to websocket

    proxy_pass http://192.168.1.91:8093;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    proxy_set_header   host              $http_host;
    proxy_set_header   x-real-ip         $remote_addr;
    proxy_set_header   x-forwarded-for   $proxy_add_x_forwarded_for;
  }
}

Replace the server_name joha.nz to your own and 192.168.1.91:8093 to the server and port the Node.JS script is running on.

SockJS and HTTP headers

Because HTTP headers can be easely forged, sockJS removes all headers besides host, x-real-ip, x-forwarded-for and a few more. So if you want to pass on custom headers from nginx, you must edit sockjs/lib/transport.js (Session.prototype.decorateConnection).

/*
  headers = {};
  _ref = ['referer', 'x-client-ip', 'x-forwarded-for', 'x-cluster-client-ip', 'via', 'x-real-ip', 'host', 'user-agent', 'accept-language'];
  for (_i = 0, _len = _ref.length; _i < _len; _i++) {
	key = _ref[_i];
	if (req.headers[key]) headers[key] = req.headers[key];
  }
  if (headers) return this.connection.headers = headers;
*/

  this.connection.headers = req.headers;

  if (this.connection.headers) return true;

Node.JS

Use something like Sock JS to handle the websocket connection and offer backwards compatibility.
Here's some boilerplate code for Sock JS in Node.JS:

var webSocketServer = require("sockjs");
var http = require("http");
var log = require("datelog");

var httpPort = 8093;
var wsServer = webSocketServer.createServer();
var httpServer = http.createServer();


function main() { 
  httpServer.listen(httpPort);
  wsServer.installHandlers(httpServer, {prefix:'/ws'});

  wsServer.on("connection", clientConnect);
  
  log("Server started");
}


function clientConnect(connection) {
  // Incomming "websock" connection ...

  var IP = connection.remoteAddress == "127.0.0.1" ? 
      connection.headers["x-real-ip"] : connection.remoteAddress,
    protocol = connection.protocol,
    visitor = new Visitor(connection, IP),
    agent = connection.headers["user-agent"];
  
  connection.on("data", reciveMessage);
  connection.on("close", connectionClosed);

  log("Connection via " + protocol + " from " + IP);
  
  function connectionClosed() {
    log("Closed " + protocol + " from " + IP);
  }

  function reciveMessage(msg) {
    var GS = String.fromCharCode(29);
    var APC = String.fromCharCode(159);
    
    console.log(IP + ": " + msg);
    
    var arr = msg.split(APC), 
      action = arr[0], 
      data = [];
      
    if(arr.length > 1) data = arr[1].split(GS);
    
    switch (action) {
      case "DOMAIN":       visitor.visit(parseText(data[0]));    break;
      case "START_READ":   visitor.read(data[0]);                break;
      case "STOP_READ":    visitor.stopRead(data[0]);            break;
    }
  }
}

You should replace visitor with your own OOP-style module.

To send something back to the client:

connection.write("command" + APC + "hello" + GS + "world");

HTML + CSS

Build the static layout / wire-frame / template with HTML and CSS.

Separate JS and CSS by using id's for JS and class for CSS. Search your .css files and re-factor any code for lines starting with # ...

Fill the wire-frame/template with some example data for convenience, so that you can work on the design without the JavaScript.

SEO !?

Yeh, this doesn't work well for SEO. But this is a web application where you'll probably need some sort of credentials to see the data anyway! Make a separate page for your copy and marketing!

Client JavaScript code

To make the client work with sockJS you need to include the sockJS client code:

<script src="../..//cdn.jsdelivr.net/sockjs/0.3.4/sockjs.min.js"></script>

Then the boilerplate code for the client look something like this:

var connection = {readyState: false}; // Connection to server

function connect(callback) {
  var server = "http://joha.nz/ws";
  
  console.log("Connecting to " + server + " ...");

  // Close the connection if already connected
  if(connection.readyState) {
    connection.close();
  }
  
  connection = new SockJS(server, '', {debug: true});
  connection.onopen = function() {
    console.log("connection open");
    
    if(callback) callback();
  };
  connection.onmessage = function(e) {
    console.log("Server message: " + e.data);
  };
  connection.onclose = function() {
    console.log("connection closed");
  };
}

function connSend(msg) {
  var open = 1;
  if(connection.readyState==open) {
    console.log("Sending: " + msg);
    connection.send(msg);
  }
  else {
    console.log("Not connected to server! Unable to send message.");
  }
}

Then use JavaScript to populate tables, add objects to the canvas scene or whatever.
Managing the DOM with JavaScript is tedious, but don't be tempted to use .innerHTML for more then one-value fields, or you'll loose all the advantages, like adding event listeners and front-end prototypes (classes) for abstraction. All HTML code should go into the .html files!

You should have a function that parse the server messages. You can then use that function to feed the app for self testing, independent from the back-end.

Database

JavaScript wasn't really designed to work with databases. But you'll need one if you want to keep the data between server reboots. The simplest form would be to use JSON.stringify to serialize and save to file and then JSON.parse to deserialize. There are however plenty of No-SQL databases that works well with Node.JS and my own database abstraction module if you want to use a SQL database.

Written by Johan Zetterberg April 15th 2015.